1. Field of the Invention
The present invention relates to a signal processing apparatus and method using a difference in sampling phase for removing aliasing components and to thereby broaden the frequency band of a signal and an image processing apparatus and method for processing an image signal obtained by capturing a plurality of images having a predetermined difference in sampling phase by a single imaging element.
2. Description of the Related Art
In the related art, there has been a signal processing method in a charged coupled device (CCD) or other image sensor for obtaining a broad band image doubled in the number of pixels by inputting two images shifted by a pitch of exactly half of the sampling interval and up-sampling and adding the two. The signal processing method is applicable to a black and white CCD image sensor, triple-element RGB CCD image sensor, etc.
Also, there have been attempts to apply this to an image obtained by a single-element CCD image sensor having a color filter, currently the mainstream in video cameras and still image pickup devices (hereinafter also referred to as electronic still cameras). Since a single-element CCD image sensor has a color filter cycle of, for example, every two pixels etc., it has been considered to shift images by half of the cycle, that is, one pixel. A signal of an image captured by a single-element CCD image sensor and converted from an analog to digital format is processed in a later camera signal processor to generate an RGB signal or a luminance signal and a color difference signal using interpolation for each color or an interrelationship among the colors.
Summarizing the problem to be solved by the invention, in a single-element CCD image sensor, however, when shifting the pixels, the array of signals of the colors becomes different from the conventional array due to the increase in the number of pixels. The later camera signal processor therefore not only has to process an increased number of pixels, but also has to change the algorithm itself.
Also, when shifting the pixels by less than the pixel pitch of the CCD image sensor, there is the disadvantage that the camera signal processor has to process a signal with uneven sampling, so the processing becomes complex.
For example, when using a Bayer array single-element CCD image sensor obtaining a R (red), G (green), and B (blue) pixel array pattern shown in FIG. 12A to obtain a total of four images shifted one pixel each in the vertical, horizontal, and diagonal directions, the obtained pixel arrays of the R, G, B data become as shown in FIGS. 12E, 12F, and 12G, which are different from the normal pixel arrays shown in FIGS. 12B, 12C, and 12D.
Namely, as will be understood by comparing FIGS. 12C and 12D with FIGS. 12F and 12G, the numbers of samples of R data and B data are increased four times, while as is understood by comparing FIG. 12B and FIG. 12E, the number of samples of G data is only doubled.
Therefore, the camera signal processor has to process R, G, B data of the pixel arrays shown in FIGS. 12E, 12F, and 12G, so there is the disadvantage that the processing becomes complex.
Also, when shifting an image by less than the pixel pitch, the sampling intervals become uneven, so there is the disadvantage that the processing becomes difficult.